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NORT-0068-US 
(12403RRUS01U) 

ROUTING DATA 
TECHNICAL FIELD 

The invention relates generally to networks, and, more particularly, to 
routing signals efficiently over a network. 

BACKGROUND 

Communication networks are widely used to link various nodes, such as 
personal computers, servers, gateways, and the like. Communication networks 
may include private networks, for example, local area networks (LANs) and wide 
area networks (WANs), or public networks as in the Internet. Popular forms of 
communications across such networks include electronic mail, file transfer, web 
browsing, and other exchanges of data. 

With the increased capacity and reliability of communications networks, 
transmission of real-time information, such as voice and streaming video, over 
networks has become possible. The ability to transfer a wide variety of 
information has made communication networks a popular choice of 
communication amongst users, as evidenced by the emergence of the Internet, 
for example. The popularity of networks as an attractive mode of 
communication has resulted in increased traffic, thereby making today's 
networks more prone to congestion, particularly during peak hours. 

To avoid network congestion, developers routinely employ routing 
algorithms to determine efficient paths from source to destination nodes. 
Generally, routing algorithms perform two main functions ~ selecting routes 
between the source and destination nodes and delivering messages to the 
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intended destination after the routes have been determined. The latter function 
of delivering messages to the intended destination involves use of data 
structures known as routing tables to deliver messages. The former function of 
selecting routes typically involves a collection of routing algorithms executing at 
5 different nodes that exchange services or perhaps information. 

It is desirable that routing algorithms correctly and expeditiously 
determine efficient routes to transmit data. Inefficient routing may delay the 
delivery of data, or even cause data to be lost, which may lead to network 
10 congestion. 

^ A need thus exists for an improved method and apparatus of routing data 

# within a communications system. 

| 15 SUMMARY 

s In general, according to one embodiment, a method comprises storing a 

m first data structure containing costs associated with transmitting data between 

U routers in a network. The method comprises combining the first data structure 

0 20 with itself to determine a cost of transmitting the data. The method further 

r comprises transmitting the data along a route based on the calculated cost. 

Some embodiments may have one or more of the following advantages. 
A reliable and efficient method and apparatus are provided for routing data from 
25 a source to a destination node. In one embodiment, a data packet may be 
transmitted to its intended destination in a "cost" efficient manner, where the 
cost may be one or more of the factors including physical distance, delay, 
expense, reliability, bandwidth, load, and security between nodes. 
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Other features and advantages will become apparent from the following 
description, from the drawings, and from the claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

5 

Figure 1 is a block diagram of an embodiment of a communications 
system in accordance with the present invention; 

Figure 2 is a block diagram of a router that may be employed in the 
10 communications system of Figure 1; 

Figure 3 illustrates an exemplary data packet 250 that may be 
communicated in the communications system of Figure 1; 

15 Figure 4 is a flow diagram of a method in accordance with one 

embodiment of the present invention that may be implemented in the 
communications system of Figure 1; 

Figure 5 is a flow diagram of a method in accordance with one 
20 embodiment of the present invention that may be implemented in the 
communications system of Figure 1; and 

Figure 6 is a flow diagram of a method in accordance with one 
embodiment of the present invention that may be implemented in the 
25 communications system of Figure 1. 

DETAILED DESCRIPTION 

In the following description, numerous details are set forth to provide an 
30 understanding of the present invention. However, it will be understood by those 
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skilled in the art that the present invention may be practiced without these 
details and that numerous variations or modifications from the described 
embodiments may be possible. 

5 Referring now to Figure 1, an exemplary communications system 1 

comprising a network 3 in accordance with one embodiment of the present 
invention is illustrated. In one embodiment, the network 3 may be a packet- 
based network, such as an Asynchronous Transfer Mode (ATM) network or an 
Internet Protocol (IP) network. As utilized herein, the term "network" may be 
10 one or more communications networks, channels, links, or paths and systems or 
devices (such as routers) used to route data over such networks, channels, links, 
or paths. 

The ATM standard is established by the International Telegraph and 
15 Telephone Consultative Committee Communications (CCITT) Standards 
Organization. One version (Version 4) of IP is described in Request for 
Comments (RFC) 791, entitled "Internet Protocol," dated September 1981. 
Another version of IP is IPv6, as described in RFC 2460, entitled "Internet 
Protocol, Version 6 (IPv6) Specification," dated December 1998. In an ATM 
20 network, packets or cells are used to communicate information. 

Although not so limited, in the illustrated embodiment, the network is an 
IP network. The IP network is a connectionless packet-switched network in 
which routing of packets is based on a source and destination address carried in 
25 each packet. The packets can travel independently over any path (and possibly 
over different paths) to a destination point identified by the destination address 
carried in the packets. Thus, in an IP network, the packets may even arrive out 
of order. 
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The network 3 comprises two hosts 4-5 and six nodes 6-11. In one 
embodiment, the hosts 4-5 may be computers that are part of local area 
networks. The six nodes 6-11 are linked to each other directly, or through one 
or more nodes 6-11. For ease of illustration, only six nodes 6-11 are illustrated, 
5 although the network 3 may include fewer or more than six nodes. Similarly, the 
network 3 may include additional or fewer than two hosts 4-5. In one 
embodiment of the invention, each node 6-11 includes a router 26-31, which 
may be implemented in a computer, server, gateway, internet device, and the 
like. Internet devices may include televisions, appliances, or any other device 
10 capable of interfacing with the network 3. 

Each router 26-31 determines the next network node to which a data 
packet is forwarded before the data packet ultimately arrives at its final 
destination. As described more fully below, in accordance with one embodiment 

15 of the instant invention, one or more of the routers 26-31 in the network 3 
determine an efficient route to deliver data packets to the final destination. A 
Router 26-31 may be located at any junction of networks or in gateways, 
including at an Internet point of presence (POP). A POP is the access point to 
the Internet, and has a unique Internet address. Independent service providers 

20 or online service providers typically have a point of presence on the Internet. 

Routers 26-31 typically create or maintain a table of the available routes 
and the route conditions and use this information to determine the best route for 
a given packet. The packet may travel through a number of network nodes, and 
25 hence through several routers, before arriving at its final destination. 

As shown, the first node 6 is coupled to the second node 7 over a first link 
64, to the third node 8 over a second link 66, and to the fourth node 9 over a 
third link 68. The second node 7 is coupled to the third node 8 over a fourth link 
30 70 and the fourth node 9 over a fifth link 72. The third node 8 is coupled to the 



fifth and sixth nodes 10, 11 over the sixth and seventh links 74, 76, respectively. 
The fourth node 9 is coupled to the second node 2 over the fifth link 72, to the 
third node 8 over an eighth link 76, and to the fifth node 10 over a ninth link 78. 
The fifth node 10 is coupled to the sixth node 11 over an eleventh link 80. For 
illustrative purposes, the fourth link 72 is depicted as a bi-directional link, while 
the other links 64, 66, 68, 70, 74, 76, 78, 80 are unidirectional; however, it 
should be appreciated that the network 3 may include any combination of 
unidirectional or bi-directional links. 

Each link may have a "cost" associated with it. The "cost," as used 
herein, may, in one embodiment, represent one or more of the following: 
physical distance, delay, expense, reliability, bandwidth, load, or security 
between nodes. Physical distance is typically the sum of the distance associated 
with each link traversed, or, alternatively, may be defined in terms of "hop 
count," which refers to the number of passes through internetworking products, 
such as routers, that a packet takes en route from a source to a destination 
node. Reliability typically refers to the dependability (usually described in terms 
of the bit-error rate) of each network link. Routing delay, which may depend on 
the bandwidth of intermediate network links, the port queues at each router 
along the way, network congestion on all intermediate network links, and the 
like, usually refers to the length of time required to move a packet from a source 
to destination node. Bandwidth typically refers to the available traffic capacity of 
a link, while load refers to the degree to which a network resource, such as a 
router, is busy. Load can be calculated in a variety of ways, including central 
processing unit (CPU) utilization and packets processed per second. 
Communication expense refers commonly to the expenditure associated with 
transmitting data packets from a source to destination node. Security typically 
refers to the level of security associated with each link or path. In alternative 
embodiments, the "cost" may be any other standard of measurement upon 
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which it is desirable to determine routing. The cost may be different for different 
directions on the same link. 

For illustrative purposes, exemplary costs are shown between the nodes 
5 6-11 of the network 3. Although not so limited, the cost associated with each 
link in the illustrated embodiment represents the physical distance between any 
two particular nodes. For instance, the cost between the first node 6 and 
second, third, and fourth nodes 7, 8, 9, is two, five, and one, respectively. The 
cost between second node 7 and third and fourth nodes 8, 9 is three and two, 
10 respectively. The cost between the third node 8 and the fourth, fifth, and sixth 
nodes 9, 10, 11, is three, one, and five, respectively. The cost between the 
fourth and fifth nodes 9, 10 is one, and the two between the fifth and sixth 
nodes 10, 11. 

15 In accordance with one embodiment of the instant invention, one or more 

of the routers 26-31 determine a "cost" efficient route for data packets between 
a source and destination node. Figure 2 depicts one embodiment of a router 200 
in accordance with the present invention. While the router 200 of Figure 2 may 
be implemented at any or all of the nodes 6-11 in the network 3 of Figure 1, for 

20 ease of illustration it is herein assumed that the router 200 represents the first 
router 26 at the first node 6 of the network 3. 

The router 200 comprises first, second, and third interfaces 205, 210, 215, 
that are respectively coupled to the routers 27, 28, 29 at the second, third, and 

25 fourth nodes 7, 8, 9 (see Figure 1), respectively. A fourth interface 216 is 
provided for interfacing with the first host 4. The router 200 comprises an input 
queue 217 for receiving data packets from the first, second, third and fourth 
interfaces 205, 210, 215, 216. An exemplary structure of an IP data packet 250 
is illustrated in Figure 3. The data packet 250 comprises an IP version field 252 

30 that indicates the current IP version. A header length field 254 is provided for 
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identifying the length of the header of the IP data packet 250. The data packet 
250 comprises a type of service field 256 that allows the data packet 250 to be 
delivered based on a variety of service factors, such as delay, throughput, 
reliability, expense, and the like. Thus, it is possible to transmit the data packet 
5 250 through a route that presumably has the least amount of delay, highest 
throughput, least chance of getting lost, or least cost. 

The data packet 250 comprises a total IP length field 258 for indicating 
the total length of the data packet 250, including the IP header and the 
10 accompanying data. The maximum length for the IP data packet 250, for 
example, is 65,535 bytes (for IPv4). The data packet 250 may also include other 
fields, such as the packet ID number, fragment, and time-to-live, and IP header 
$ checksum fields 260, 262, 264, 266. A protocol carried field 268 identifies a 
hi higher layer protocol, such as Transmission Control Protocol (TCP), User 

M 15 Datagram Protocol (UDP), Internet Control Message Protocol (ICMP), and the 
ffi like. 

K The data packet 250 comprises source and target address fields 270, 272. 

2; In Version 4 of the Internet Protocol, the source and target address fields 270, 
© 20 242 of the data packet 250 are typically 4 bytes long. An options field 274 is 
provided in the data packet 250 for enabling measurement of time delay, routing 
messages, or specifying routing instructions. The options field 274, for example, 
may be utilized to specify data packet 250 route from source to destination by 
providing the intermediate routers that the data packet 250 should traverse. In 
25 Version 4 of the IP, for example, up to nine (9) intermediate routers may be 
specified between the source and destination. The data packet 250 also 
comprises a data field 276 that contains the data being transmitted. 

Referring again to Figure 2, the incoming data packets 250 are provided 
30 from the input queue 217 to an IP options block 295, which processes the 



9 



options field 274 of each received data packet 250 to determine if source routing 
is specified. In alternative embodiments, the IP options block 295 may perform 
additional functions as well. If source routing is specified in the IP options field 
274, the data packet 250 is provided to the IP output block 300, which forwards 
5 the data packet 250 to the next router specified in the options field 274 through 
the interfaces 205, 210, 215. 

If the IP options block 295 determines source routing is not specified, 
then, at block 315, the router 200 determines if the data packet 250 is intended 
10 to be forwarded, broadcast, or delivered to the first host 4 coupled to the router 
200. If the data packet 250 is to be forwarded or broadcast to other routers 27- 
^ 31 on the network 3, then the data packet 250 is provided to the IP output block 
J 300, which transmits the data packet 250 to the appropriate router or routers 

ffl 27-31. If the data packet 250 is not intended to be forwarded or broadcast to 
^ 15 other routers 27-31, it may be provided to a TCP or UDP layer for delivery to the 
f: first host 4 through the interface 216, or, alternatively, it may be delivered to an 
Internet Control Message Protocol (ICMP) block 325. The ICMP block 325 
|:i typically handles error and control messages for IP. 

w 20 The router 200 includes a routing table 330 that in one embodiment 

contains information to aid the process of path determination. Routing 
information stored in the routing table 330 may vary, depending on the routing 
algorithm employed. The routing table 330 may include destination (next hop) 
associations that indicate to the router 200 that a particular destination may be 
25 reached efficiently by transmitting the data packet 250 to a particular router 
representing the "next hop" on the way to the final destination. In one 
embodiment, the routing table 330 may contain information about the "costs", in 
terms of delay, distance, and the like, that are associated with transmitting data 
packets 250 between routers. 



30 
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Routers 26-31 (see Figure 1) routinely communicate with one another and 
exchange messages to build a substantially detailed picture of network topology. 
The routing table 330 of the router 200 may be updated by a route command 
340, routing daemon 335, or an ICMP redirect 325. The route command 340 to 
5 update the routing table 330 may be issued by an operator sitting at the host 4, 
for example. The routing daemon 335 is a software application that is capable of 
executing Routing Information Protocol (RIP), which is commonly used to 
periodically exchange routing information from other computers on the network 
3 {e.g., about every 30 seconds). 

10 

In accordance with one embodiment of the present invention, a route 
calculation block 345 of the router 200 uses the information from the routing 
table 330 to calculate a "cost" efficient route for the packet data 250. One 
embodiment of a routing algorithm employed by the route calculation block 345 
15 is described in more detail below. The route calculation block 345 determines an 
efficient route for delivering the packet data 250 and provides the routing 
information to the IP output block 300, which then forwards the packet data 250 
to the next router based on the route determined by the route calculation block 
345. 

20 

Selected portions of the router 200, such as the routing calculation block 
345, may be implemented in hardware, software, or a combination thereof. In 
one embodiment, the router 200 may include software routines, modules, or 
drivers that may be executable on the control unit 245. Data instructions 
25 associated with the software routines may be stored in the storage unit 360. 

Referring now to Figure 4, one method in accordance with the present 
invention of delivering data packets 250 efficiently over the network 3 based on 
one or more of the "costs" is illustrated. For illustrative purposes, the method of 
30 Figure 4 is described in the context of transmission of the data packet 250 from 
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the first host 4, which is coupled to the first router 26, to the second host 5, 
which is coupled to the sixth router 31. The method of Figure 4 begins at block 
450, where the first router 6 receives the data packet 250 from the first host 4 
for delivery to the second host 5. Accordingly, the source address field 270 
includes the IP address of the first host 4, and the target address field 272 
includes the IP address of the second host 5. At block 460, the first router 26, 
based on the type of service field 256 of the data packet 250, determines the 
type of service that has been designated by the first host 4. That is, the first 
host 4 may, in one embodiment, designate that the data packet 250 be delivered 
to the second host 5 using the shortest route. In alternative embodiments, the 
first host 4 may designate that the packet data 250 be delivered using the least 
expensive route or the most reliable route, for example. 



At block 470, the first router 26 calculates an efficient route based on the 
type of service designated by the first host 4. Figure 5 illustrates one 
embodiment of the block 470. The method of Figure 5 begins at block 510, 
where the route calculation block 345 of the first router 26 generates a 
"connection" data structure of the network 3. Although a variety of data 
structures may be utilized, in the illustrated embodiment the data structure 
utilized herein is a matrix. A connection matrix is provided that is representative 
of the network topology, where the connection matrix illustrates the adjacent 
nodes to each node. An exemplary connection matrix for the network 3 of 
Figure 1 is as follows: 

11110 0 
0 1110 0 
0 0 10 11 
0 11110 
0 0 0 0 1 1 
0 0 0 0 0 1 



Connection Matrix C = 
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Since the network 3 of Figure 1 comprises six nodes 6-11 (see Figure 1), 
the above connection matrix has six rows and six columns. Each row of the 
connection matrix illustrates the adjacent nodes of each node. Numerals "1" and 
"0" denote whether nodes are adjacent to each other, in particular, numeral "1" 
5 denotes that two particular nodes are adjacent, while numeral "0" indicates no 
adjacency. For example, the first row, which comprises numerals "11110 0," 
indicates that the first node 6 is adjacent to the second, third, and fourth nodes 
7, 8, 9, but not to the fifth and sixth nodes 10, 11. Likewise, the second row, 
which comprises sequence "011110 0," indicates that the second node 7 is 
10 adjacent to the third and fourth nodes 8, 9, but not to the first, fifth and sixth 
nodes 6, 10, 11. Similarly, rows third, fourth, fifth, and sixth rows depict the 
adjacent nodes for respective third, fourth, fifth, and sixth nodes 8-11. 

The route calculation block 345 (see Figure 2) may generate the data for 

15 the connection matrix in a variety of ways, including generating based on the 
information stored in the routing table 330. The routing table 330, in one 
embodiment, may include information regarding nodes that are adjacent to the 
first router 26. For example, the first router 26 may obtain routing table 
information about its adjacent nodes by submitting a layer 1 (Ethernet) 

20 broadcasting request to each of its interfaces 205, 210, 215. The routing table 
330 may use routing protocols, such as Open Shortest Path First (OSPF) and 
Routing Information Protocol (RIP), to obtain data for the other rows. For 
example, the first router 26 may utilize "flooding" to acquire data for the other 
rows of the connection matrix. Flooding involves each router 26-31 periodically 

25 broadcasting its adjacent nodes to all its neighboring nodes. The neighboring 
node in turn forwards the received broadcast message, along with the 
information about its own neighboring nodes, to all of its neighbors. 
Accordingly, the routing table 330, in one embodiment, comprises information 
about the adjacent nodes of the first router 26, as well as about the adjacent 

30 nodes of other nodes in the network 3. The route calculation block 345, in one 
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embodiment, accesses the routing table 330 to generate the connection matrix 
for the network 3. 

At block 520, the route calculation block 345 generates a "cost" matrix. 
5 In the illustrated embodiment, the cost matrix is representative of the distance 
between nodes in the network 3. An exemplary cost {e.g., distance) matrix for 
the network 3 of Figure 1 is depicted below: 

D 00 00 
D 2 00 00 

00 D D 5 

1 D oo 
oo 1 D 2 

00 00 1 

10 

As shown, although not so limited, in the illustrated embodiment, the cost 
factor is expressed as exponents for reasons explained in more detail later. In 
the illustrated embodiment of the cost matrix, the letter 'D' is a variable (without 
value) that stands for distance. The cost matrix has six rows and six columns 

15 since the network 3 of Figure 1 has six nodes 6-11. Each row of the cost matrix 
depicts the minimum distance between a particular node and the remaining 
nodes in one step {i.e., hop). That is, for example, the first row depicts the 
minimum one-step distance between the first node 6 and the remaining nodes 7- 
11, while the second row depicts the minimum one-step distance between the 

20 second node 7 and the remaining nodes 1, 8-11, and so forth. 

The first row, which comprises the sequence "1 D 2 D 5 D oo oo," indicates 
that the minimum distance (expressed as exponents) in one step between the 
first node 6 and second, third, fourth nodes 7, 8, 9, is respectively 2, 5, and 1. 
25 The minimum one-step distance between the first node 6 and fifth and sixth 



Distance Matrix D = 
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D 2 
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00 
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D 3 


00 


00 
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00 


D 2 


D 3 


00 


00 


00 


00 


00 


00 
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nodes 10, 11 is infinity, since no direct connection exists between the first node 
6 and fifth and sixth nodes 10, 11. The first element of the first row represents 
the distance between the first node 6 to itself. In this case, the first element of 
the first row is one because expressing the distance {i.e., zero) between the first 
5 node 6 and itself as an exponent yields one (/.e, D° equals 1). Similarly, the 
fourth row the cost matrix, which comprises the sequence "oo D 2 D 3 1 D oo," 
illustrates that the minimum one-step distance between the fourth node 9 and 
the second, third, and fifth nodes 7, 8, 10 is 2, 3, 1, respectively. The fourth row 
also illustrates that no one-step connection exits between the fourth node 9 and 
10 first and sixth nodes (as denoted by the symbol oo) 6, 11, while the distance 
between the fourth node 9 and itself is zero (denoted as one since DO equals 1). 

|j The route calculation block 345 may, in one embodiment, generate data 

51 for the cost matrix in a similar way as the data for the connection matrix is 

S 15 generated. That is, flooding may be utilized to gather data for the cost matrix 

SH between each of the neighboring nodes in the network 3. Any other available 

■: t 2 

s means for obtaining the data for the cost matrix may be employed without 

•jr.v.l:', 

h deviating from the spirit of the instant invention. 

P 20 At block 530, the route calculation block calculates a next_step matrix. 

The next_step matrix in one embodiment is calculated by combining the cost 
matrix with itself. The cost matrix is combined with itself in one embodiment 
using the following mathematical function: 

25 next_step cost matrix (D 2 ij) = mink=i to 6 (D ik * D k j), 

where i indicates the row and j indicates the column of the cost matrix (D), and k 
ranges from 1 to the number of nodes {e.g., six in the illustrated embodiment) in 
the network. The cost matrix, as described above, illustrates the minimum one- 
30 step distance between node i (row) and node j (column). In accordance with 
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one embodiment of the present invention, combining the cost matrix with itself 
once using the above mathematical function yields a resultant matrix that 
illustrates the minimum distance between a particular node and the other nodes 
using two or fewer steps. The resultant next_step data structure in the illustrate 
embodiment is: 



D D D 
1 D 3 D 2 

00 1 CO 



oo D D 1 



1 

00 
CO 



D D 

D oo 
2 

00 oo 1 D 



CO 00 00 00 00 



1 D 

oo 1 

00 CO 

2 

oo D 



D 5 D 
D 3 D 2 



1 



00 

1 



00 00 

D D 5 



00 00 00 00 



00 00 00 CO CO 



D D D D D 



10 



30 00 

2 

oo D 



CO 00 00 00 



D 
D 
D 

1 



The above mathematical function is similar to ordinary matrix 
multiplication except, instead of summation, the minimum function is employed. 

10 As an example, to obtain the first element {i.e., first row, first column), the first 
row ("1 D 2 D 5 D oo oo' 7 ) of the cost matrix is multiplied with the first column (*1 
oo oo oo oo oo") of the cost matrix, and instead of summing each element, the 
smallest {i.e., minimum) element is selected as the first element. That is, the 
smallest element is selected from the following elements [1, (i.e., lxl), oo {i.e., 

15 D 2 x oo), oo (/a, D 5 x oo), oo (i.e., D x oo), oo (i.e., oo x oo), oo (i.e., oo x oo)], 
which in the instant example is one (1). As an added example, to obtain the 
element at row 2, column 4, the second row f oo l D 3 D 2 oo oo") of the cost 
matrix is multiplied with the fourth column CD D 2 oo 1 oo oo") of the cost matrix, 
and the smallest element from the following elements [oo (i.e., oo x D), D 2 (i.e., 

20 1 x D 2 ), oo (i.e., D 3 x oo), D 2 (i.e., D 2 x 1), oo (i.e., oo xoo), oo (i.e., oo x oo)] is 
selected. In this case, the smallest element is D 2 . Other elements of the D 2 j 
cost matrix are calculated similarly. 



Since the elements of the matrix are either exponents of D or infinity, the 
25 multiplication reduces to exponent summation, which generally is a relatively 
quick and simple operation. Furthermore, since the minimum distance between 
any two nodes can only be one, the mathematical operation over k iterations can 
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stop as soon as a resulting element of D having a value of one is found. Thus, 
the summation over k products results in replacing the infinity distance by the 
smallest exponent summation until either the result is one (1) or k is exhausted. 

At block 550, the routing calculation block 345 determines if the next_step 
cost matrix is equal to the matrix of the previous step. Accordingly, in the 
illustrated embodiment, at the block 550, the D 2 ij cost matrix is compared to the 
previous cost matrix, Dg. The two matrixes are equal if all of the corresponding 
elements of the matrices are equal. Since the D 2 u cost matrix and the Dy matrix 
in the illustrated embodiment are not equal, at the block 530, the next_step cost 
matrix is re-calculated. If, at the block 550, the resultant matrix is equal to the 
matrix of the previous step, then it is an indication that no more processing is 
required because the resultant matrix represents the minimum distance between 
any two nodes for any given number of steps. 

At the block 530, the next_step cost matrix {e.g., D 3 jj cost matrix) is 
calculated using the following mathematical function: 



D 3 ij = min k =ito6(Dik, D\,), 



where i indicates the row and j indicates the column of the cost matrix (Djj), D 2 y 
is the previous step matrix calculated above, and k ranges from 1 to the number 
of nodes in the network. The resultant next_step cost matrix in the illustrated 
embodiment is: 
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The resultant matrix, D 3 j, illustrates the minimum distance between any 
two nodes (node i (row) and node j (column)) in three or fewer steps. At the 
block 530, the resultant next_step cost matrix {e.g., D 3 j cost matrix) is compared 
to the previous step cost matrix {e.g., D 2 ij cost matrix) to see if the two matrices 
are equal. And, since in the instant case the two matrices are not equal, the 
next_step cost matrix, at the block 530, is computed once again. 

At the block 530, the next_step cost matrix {e.g., D 4 j cost matrix) is 
calculated using the following mathematical function: 

D 4 ij = min k =ito6(Dij, D 3 ig), 



where i indicates the row and j indicates the column of the cost matrix (Djj), D 3 jj 
is the previous step matrix calculated above, and k ranges from 1 to the number 
15 of nodes in the network. The resultant next_step cost matrix in the illustrated 
embodiment is: 
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20 The resultant matrix, D 4 j, illustrates the minimum distance between any 

two nodes (node i (row) and node j (column)) in four or fewer steps. At the 
block 530, the resultant next_step cost matrix {e.g., D 4 j cost matrix) is compared 
to the previous step cost matrix {e.g., D 3 j cost matrix) to see if the two matrices 
are equal. And, since all of the elements of the two matrices are equal, it is an 

25 indication that previous step matrix, D 3 j, illustrates the minimum distance 
between any two nodes for any given number of steps. 
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Upon calculating the minimum distance between any two nodes within the 
network, the routing calculation block 345, at block 550, determines the efficient 
route or path the data packet 250 takes by determining any or all of the 

5 intermediate nodes between the source node {e.g., first node 6) and destination 
node {e.g., sixth node 11). One embodiment of the block 550 is illustrated in 
Figure 6. For illustrative purposes, the method of Figure 6 is described herein 
with reference to a specific example of determining the shortest distance from 
the first node 6 to the sixth node 11. The method of Figure 6 begins at block 

10 610, where the routing calculation block 345 determines the minimum distance 
between a current pair of nodes. In one embodiment, during the first iteration, 
the current pair of nodes represents the source and destination nodes, which in 
the instant example includes the first and sixth nodes 6, 11. In one 
embodiment, the minimum distance is determined with reference to the element 

15 at row one, column six of the cost matrix, D 3 ij, which, as stated earlier, illustrates 
the minimum distance between any two nodes for any given number of steps. 
In this example, the minimum distance is four (4) between the first and sixth 
nodes 6, 11. 

20 At block 620, the routing calculation block 345 determines at least one 

route/path having the minimum distance of four between the current pair {e.g., 
first and sixth nodes 6,11) of nodes. In one embodiment, to determine the 
route/path having the minimum distance of four, the IP route block 550 employs 
the mathematical function shown in equation (1) below. 

25 

D 3 (F, S)min = min k=1 to e (D 3 (F)(k) * D 3 (k)(S)), (1) 

where F and S represent respective first and second input nodes, and D 3 j is the 
cost matrix illustrating the minimum distance between any two nodes for any 
30 given number of steps. Equation (1) essentially determines the minimum 
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distance between a source node and destination node and all of the intermediate 
nodes. Thus, in the illustrative example (where F=l and S=6), equation (1) 
essentially determines the distance from the first node 6 to the sixth node 11, 
then distance from the first node 6 to the second node 7 and between the 
second node 7 to the sixth node 11, then the distance from the first node 6 to 
the third node 8 and third node 8 to the sixth node 11, then the distance from 
the first node 6 to the fourth node 9 and fourth node 9 to the sixth node 11, and 
so forth. 

As mentioned above, in the illustrative example, F is one, and S is six. 
Thus, employing the mathematical expression of equation (1) to the illustrated 
example yields the terms [(1)(D 4 )=D 4 , (D 2 )(D 5 )=D 7 , (D 4 )(D 3 )=D 7 , (D)(D 3 )=D 4 , 
(D 2 )(D 2 )=D 4 , (D 4 )(D 1 )=D 4 ] / which indicates that, aside from the first and sixth 
nodes 6, 11, the fourth and fifth nodes 9, 10 involve the shortest distance, 
namely 4. This means that that two possible minimum-distance routes from the 
first node 6 to the sixth node 11 are either: 1^5-^6 OR 1-^4^6. 

Assuming that the route/path under consideration is 1^ 5 -> 6, at block 
640, the routing calculation block 345 determines if a direct connection exists 
between a first pair of nodes in the at least one route {e.g., l-> 5 -> 6). In the 
instant example, the first pair of nodes includes the first and fifth nodes 6, 10. 
In one embodiment, the connection matrix (C) depicted above is utilized to 
determine if a direct connection exists between the first pair. The element at 
row one, column five of the connection matrix is zero, which indicates that no 
direct connection exists between the first and fifth nodes 6, 10. Thus, at block 
650, the routing calculation block 345 uses the next route {e.g., 1 -> 4 6) of 
the at least one route that was calculated at the block 620. 

Assuming that the route/path now under consideration is 1 -> 4 -> 6, at 
the block 640, the routing calculation block 345 determines if a direct connection 
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exists between a first pair of nodes in the at least one route {e.g., 1 -> 4 -> 6). 
In the instant example, the first pair of nodes includes the first and fourth nodes 
6, 9. Using the connection matrix, the routing calculation block 345 determines 
that a direct connection exists between the first and fourth nodes 6, 9 since the 
5 element at row one, column four is one. 

At block 660, the routing calculation block 345 determines if any 
additional pair of nodes exists in the at least one route {e.g., 1 -> 4 6) 
calculated at the block 620. Since another pair (4 and 6) of nodes does exist in 
10 the at least one route, at block 670, the routing calculation block 345 determines 
if a direct connection exists between the fourth and sixth nodes 9, 11. The 
routing calculation block 345 checks to see the value of the element at row four, 
% column six to determine if a direct connection exists between the fourth and 
sixth nodes 9, 11. Since the value of element at the row four, column six is zero, 
Sj 15 no direct connection exists between these two nodes, which means that, at block 
m 680, the variable "current pair of nodes" is set to equal the next pair of nodes 

(v {e.g., 4 and 6), and the process of Figure 6 is once again repeated. 

§J At the block 610, the routing calculation block 345 determines the 

%_ 20 minimum distance between the current pair of nodes {e.g., nodes 4 and 6). The 
P minimum distance is determined with reference to the element at row four, 
column six of the cost matrix, D 3 jj, which, as mentioned before, illustrates the 
minimum distance between any two nodes for any given number of steps. In 
this example, the minimum distance is four (3) between the fourth and sixth 
25 node 9, 11. 

At the block 620, the routing calculation block 345 determines at least one 
route/path having the minimum distance of three between the current pair of 
nodes {e.g., fourth and sixth nodes 9,11). In one embodiment, to determine the 
30 route/path having the minimum distance of three, the routing calculation block 
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345 employs the mathematical function shown above in equation (1). In this 
case, F is four, and S is six. Thus, employing the mathematical expression of 
equation (1) to the illustrated example yields the terms [(<»)(D 4 )=oo, 
(D 2 )(D 5 )=D 7 , (D 3 )(D 3 )=D 6 , (1)(D 3 )=D 3 , (D)(D 2 )=D 3 , (D 3 )(1)=D 3 ], which 
5 indicates that, aside from the fourth and sixth nodes 9, 11, the fifth node 10 
involves the shortest distance, namely 3. This means that the minimum-distance 
route from the fourth node 9 to the sixth node 11 is 4-> 5 -> 6. 

Assuming that the route/path now under consideration is 4 -> 5 -> 6, at 
10 the block 640, the routing calculation block 345 determines if a direct connection 
exists between a first pair of nodes in the at least one route (e.g., 4 -> 5 -> 6). 
i. In the instant example, the first pair of nodes includes the fourth and fifth nodes 

It 9, 10. Using the connection matrix, the routing calculation block 345 determines 
% that a direct connection exists between the fourth and fifth nodes 9, 10, since 

% 15 the element at row four, column five is one. 

f At the block 660, the routing calculation block 345 determines if any 

j!;; additional pair of nodes exists in the at least one route (e.g., 4 5 -> 6) that 

was calculated at the block 620. Since another pair (5 and 6) of nodes exist in 
|l 20 the at least one route, at the block 670, the routing calculation block 345 
¥ determines if a direct connection exists between the fifth and sixth nodes 10, 11. 

Since the element at row five, column six of the connection matrix is one, the 

route calculation block 345 determines that a direct connection exists between 

the fifth and sixth nodes 10, 11. 

; 25 

At the block 660, the routing calculation block 345 again determines if any 
more pair of nodes exists in the at least one route (e.g., 4 -> 5 -> 6). In the 
instant case since no more pairs exits, at block 690, the routing calculation block 
345 calculates, based on the earlier iterations, the minimum distance route/path 
30 between the first and sixth nodes 6, 11. That is, in the instant example, it is 
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determined that the optimal route in terms of the minimum distance {e.g., 4) 
between the first node 6 and sixth node 11 is from the first node 6 to the fourth 
node 9 to the fifth node 10 to the sixth node 11 {e.g., 1 -> 4 -> 5 •» 6). 

5 Referring again to Figure 4, at block 480, the IP output block 300 updates 

the IP options field 274 of the data packet 250 based on the efficient route 
calculated at the block 470. Thus, in the illustrated embodiment, since the 
shortest distance from the first host 4 to the second host 5 is through the first 
node, fourth node, fifth node, and sixth node, the IP output block 300 of the first 
10 router updates the IP options field 274 with the IP addresses corresponding to 
the fourth, fifth, and sixth nodes 9, 10, 11. In one embodiment, upon writing to 
the IP options field 274, the IP output block 300 of the first router 26 indicates 
that source routing is desired. 

15 At block 490, the data packet 250 is transmitted to the next router, which 

in the illustrated embodiment is the fourth router 29. When the data packet 250 
reaches the fourth router 29, it is queued and then processed by the IP options 
processing block 295 of the fourth router 29. And since the first router 26, at 
the block 480, indicates that source routing is desired, the IP options processing 

20 block 295 of the fourth router 29 provides the data packet 250 to its IP output 
block 300. The IP output block 300 then updates the IP options field 274 to 
reflect that the next destination of the data packet 250 is the fifth node 10. The 
above-described process continues until the data packet 250 is ultimately 
delivered to the second host 5. 

25 

In accordance with one embodiment of the present invention, instead of 
using source routing, each router that receives the data packet 250 may 
independently calculate a cost efficient path using the method of Figure 5 for 
delivering the data packet 250 to its final destination. That is, upon calculating 
30 the next node, the router transmits the data packet 250 without indicating the 
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use of source routing, thereby leaving it up to the receiving router to calculate 
the next destination. Without the use of source routing, each router may 
independently use the method of Figure 5 for calculating an efficient route to 
deliver the data packet 250 to its final destination. 

5 

In accordance with one embodiment of the present invention, instead of 
determining the connection and/or cost matrices, the router 200 (see Figure 2) 
may simply receive, and perhaps store, the matrices that might have been 
determined elsewhere. Thus, in one embodiment, upon receiving the connection 
10 and/or cost matrices, the router 200 calculates a cost efficient route from a 
source node to a destination node. 

Although in the illustrated embodiment the connection and cost matrices 
are generated based on adjacent nodes, such matrices are not limited as such. 
15 That is, the connection and/or cost matrices may be based on any combination 
of nodes or routers {e.g., even/odd nodes or routers) in the network 3. 

Some embodiments may have one or more of the following advantages. 
A reliable and efficient method and apparatus are provided for routing signals 

20 from a source to a destination node. In one embodiment, a data packet may be 
transmitted to its intended destination in a "cost" efficient manner, where the 
cost may be one or more of the factors such as the physical distance, delay, 
expense, reliability, bandwidth, load, and security between nodes. Accordingly, 
with the advent of the present invention, routers may, in one or more 

25 embodiments, transmit data packets to their final destination expeditiously and 
accurately, thereby improving network congestion. 

The various system layers, routines, or modules may be executable 
control units (such as control unit 355) (see Figure 2). Each control unit may 
30 include a microprocessor, a microcontroller, a processor card (including one or 
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more microprocessors or controllers), or other control or computing devices. The 
storage devices referred to in this discussion may include one or more machine- 
readable storage media for storing data and instructions. The storage media 
may include different forms or memory including semiconductor memory devices 
such as dynamic or static random access memories (DRAMs or SRAMs), erasable 
and programmable read-only memories (EPROMs), electrically erasable and 
programmable read-only memories (EEPROMs) and flash memories; magnetic 
disks such as fixed, floppy, removable disks; other magnetic media including 
tape; and optical media such as compact disks (CDs) or digital video disks 
(DVDs). Instructions that make up the various software layers, routines, or 
modules in the various systems may be stored in respective storage devices. 
The instructions when executed by a respective control unit cause the 
corresponding system to perform programmed acts. 

In the loading or transport process, data signals that are embodied in 
carrier waves (transmitted over telephone lines, network lines, wireless links, 
cables, and the like) may communicate the code segments, including 
instructions, to the terminal. Such carrier waves may be in the form of electrical, 
optical, acoustical, electromagnetic, or other types of signals. 

While the invention has been disclosed with respect to a limited number of 
embodiments, those skilled in the art will appreciate numerous modifications and 
variations therefrom. It is intended that the appended claims cover such 
modifications and variations as fall within the true spirit and scope of the 
invention. 
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What is claimed is: 



1 1. A method, comprising: 

2 storing a first data structure containing costs associated with 

3 transmitting data between routers in a network; 

4 combining the first data structure with itself to determine a cost of 

5 transmitting the data; and 

6 transmitting the data along a route based on the calculated cost. 

1 2. The method of claim 1, further comprising storing a second data 

2 structure defining router connections in the network. 

43 1 3. The method of claim 2, wherein storing the second data structure 

m 2 comprises storing a matrix defining router connections. 

§3 l 4. The method of claim 3, wherein storing the first data structure 

= 2 comprises storing a matrix, wherein the costs are based on at least one of a 

JK 3 distance, reliability, security, or expense of transmitting the data between routers 

J; 4 in the network. 

r 1 5. The method of claim 4, wherein combining the first data structure 

2 with itself calculates the cost of transmitting the data between a source router 

3 and destination router in the network for a given number of steps at minimal 

4 cost. 

1 6. The method of claim 5, wherein the transmitting the data along the 

2 route further comprises determining the route between the source router and the 

3 destination router based on the cost and the connection matrix. 
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1 7. The method of claim 2, further including determining the second 

2 data structure. 

1 8. The method of claim 1, wherein transmitting the data comprises 

2 transmitting an IP data packet. 

1 9. The method of claim 1, further including determining the first data 

2 structure. 

1 10. An apparatus, comprising: 

2 an interface adapted to receive a data packet; 

3 at least one storage device to store: 

Ji 4 a first data structure defining router connections in a 

tl 5 network; and 

CO 6 a second data structure that defines a cost associated with 

£p 7 links between routers in the network; and 

8 a controller adapted to: 

jj 9 combine the second data structure with itself at least once 

M* io to determine a cost for transmitting the data packet; and 
Pi 11 determine a route based on the first data structure and the 

¥ 12 calculated cost for transmitting the data packet. 

1 11. The apparatus of claim 10, wherein the first data structure 

2 comprises a first matrix that defines the router connections in the network 

3 wherein the router connections comprise adjacent router connections. 

1 12. The apparatus of claim 11, wherein the second data structure 

2 comprises a second matrix that defines the cost associated with each link 

3 between adjacent routers as exponents. 
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1 13. The apparatus of claim 12, wherein the cost of each link between a 

2 router and itself is defined as zero and the cost for each link from a router to a 

3 non-adjacent router is defined as infinity. 

1 14. The apparatus of claim 13, wherein the controller is adapted to 

2 combine the second matrix using the formula mini to k (D,k * Dkj), wherein k is the 

3 number of the routers and the second matrix is represented by D that has i rows 

4 and j columns. 

1 15. The apparatus of claim 14, wherein the controller is adapted to 

2 increment k if a resulting element from combining the second matrix is one. 

jj 1 16. The apparatus of claim 12, wherein the costs are based on at least 

% 2 one of a distance, reliability, security, or expense of transmitting the data packet 

g 3 between the adjacent routers in the network. 

1 17. The apparatus of claim 12, wherein the controller is further 

J;° 2 adapted to combine the second matrix with itself a number of times until the 

N= 3 cost of transmitting the data packet between a source router and destination 

p 4 router is minimum for a given number of steps. 

1 18. The apparatus of claim 10, wherein the controller is adapted to 

2 determine a direct connection between each link of the route based on the first 

3 data structure. 

1 19. The apparatus of claim 10, wherein the controller is further 

2 adapted to transmit the data packet along the route. 

1 20. The apparatus of claim 10, wherein the data packet is an IP data 

2 packet. 



28 



1 21. An article comprising at least one machine-readable storage media 

2 containing instructions for routing a data packet, the instructions when executed 

3 causing a controller to: 

4 represent node connections in a network in a first matrix; 

5 represent costs of transmitting the data packet between each of a 

6 plurality of nodes in a second matrix; and 

7 determine a route to transmit the data packet based on the first matrix 

8 and the second matrix. 

1 22. The article of claim 21, wherein the instructions when executed 

2 cause the processor to transmit the data packet over the route. 

1 23. The article of claim 21, wherein the instructions when executed 

2 cause the processor to represent adjacent node connections in the first matrix. 

1 24. The article of claim 21, wherein the instructions when executed 

2 cause the processor to represent the costs as exponents in the second matrix. 

1 25. The article of claim 24, wherein the instructions when executed 

2 cause the processor to represent a cost between each node and itself as zero 

3 and each node to a non-adjacent node as infinity. 

1 26. The article of claim 25, wherein the instructions when executed 

2 cause the processor to combine the second matrix using the formula mini to k (Djk 

3 * Dig), wherein k is the number of the routers and the second matrix is 

4 represented by D that has i rows and j columns. 
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1 27. The apparatus of claim 26, wherein the instructions when executed 

2 cause the processor to increment k if a resulting element from combining the 

3 second matrix is one. 

1 28. The article of claim 21, wherein the instructions when executed 

2 cause the processor to represent the costs comprises the processor to represent 

3 at least one of a distance, reliability, security, or expense of transmitting the data 

4 packet between each of the plurality of nodes. 

1 29. The article of claim 21, wherein the instructions when executed 

2 cause the processor to combine the second matrix with itself a number of times 
p 3 until the costs of transmitting the data packet between a source node and 
1{ 4 destination node are minimum for a given number of steps. 



yi 1 30. The article of claim 21, wherein the instructions when executed 

% 2 cause the processor to determine the route to transmit an IP data packet. 

19 1 31. A data signal embodied in a carrier wave comprising instructions for 

h 2 routing data packet to at least one of a plurality of network entities, the 

|f 3 instructions when executed causing a controller to: 

4 store a connection matrix indicating adjacent nodes in a network; 

5 store a cost matrix expressing transmission costs as exponents; 

6 and 

7 determine a route for transmitting the data packet based on the 

8 connection and cost matrices from a first node to a second node. 

1 32. The data signal of claim 31, wherein the instructions when 

2 executed cause the processor to transmit the packet data over the route. 
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1 33. A communication system, comprising: 

2 a source entity adapted to transmit a data packet; 

3 a router capable of receiving the data packet, the router adapted 

4 to: 

5 define a cost matrix containing transmission costs associated 

6 with routing the data packet between a pair of routers in a network; 

7 determine a transmission cost of transmitting the packet 

8 data to a destination entity based on the cost matrix; and 

9 transmit the data packet to the destination entity using a 

10 route associated with the transmission cost. 

1 34. The communications system of claim 33, wherein the data packet is 

2 an IP data packet. 
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ABSTRACT OF THE DISCLOSURE 



A method is provided that comprises storing a first data structure 
containing costs associated with transmitting data between routers in a network. 
The method includes combining the first data structure with itself to determine a 
cost of transmitting the data. The method further comprises transmitting the 
data along a route based on the calculated cost. 
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and the national or PCT International filing date of this application: 



(Application Number) Filing Date (Status-patented, pending, 

abandoned) 

I hereby appoint Timothy N, Ttop, Reg, No. 28,994; Fred G. Pruner, Jr., Reg. No, 40,779, Dan C 
Hu, Reg- No. 40,025; and Ruben S. Bains, Reg. No, 46,532; my patent attorneys, of TROP, 
PRUNER & HU, PC with offices located at 8554 Katy Freeway, Ste, 100, Houston, TX 77024, 
telephone (713) 468-8880, and John D. Crane, Reg. No. 2531; Howard R. Greenberg, Reg, No. 
26,171; W. Glen Johnson, Reg. No. 39,525; Randall Mishler, Reg. No, 42,006; Kevin L. Smith, 
Reg. No. 38,620; Vernon E. Williams, Reg, No. 38,713; Thomas A Gigliotti, Reg. No. 37,579; 
Eric R Jensen, Reg. No, 37,647; L Erik Fako, Reg. No. 42,522; John H. Vynalefc Reg, No. 
37 7 254; John R, Witcher, IE, Reg, No, 39,877; and R. Todd Morgan, Reg, No. 43,815, my patent 
attorneys, of Nortel Networks Limited; with full power of substitution and revocation* to prosecute 
this application and to transact all business in the Patent and Trademark Office connected herewith. 

Send correspondence to Ruben S. Bains, TROP, PRUNER & HU, P.C, 7 8554 Katy Freeway, Ste, 
100, Houston, TX 77024 and direct telephone calls to Ruben S. Bains, (713) 468-8880, 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willftil false statements may jeopardize the validity of the application or any patent issued 
thereon. 
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Full Name of Sole/First Inventor; 






FARUK S. HADZIOMEROVIC 






Inventor's Signature: jp- J^ <jt ^ ( p_^ 


Date: j 

w 




Residence: 


Citizenship: 


1702 Woodoak Dr 7 Richardson, TX 75082 


CANADA 


Post Office Address: 






1702 Woodoak Dr, ? Richardson, TX 75082 






Full Name of Second/Joint Inventor: 


Inventor's Signature; 


Date: 


Residence: 


Citizenship: 


Post Office Address: 


Full Name of Third/Joint Inventor: 


Inventor's Signature: 


Date: 


Residence: 


Citizenship: 


Post Office Address: 
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United States Patent & Trademark Office 

Office of Initial Patent Examination — Scanning Division 




Application deficiencies were found during scanning: 

□ Page(s) of pre\\ 'tsAif\afi4 rtrrt^nAm&fl^ were not present 

for scanning. (Document title) 



□ Page(s) of were not present 

for scanning. (Document title) 



n Scanned copy is best available. 



